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ABSTRACT 


This  final  report  describes  the  research  carried  oat  by  Professors 
Sanjoy  K.  Hitter  and  Bernard  Levy  and  Hr.  Tehuda  Avniel  and  Hr.  Saul  Gelfand 
daring  the  time  period  Harch  15,  1984  to  Harch  15,  198JJ7  with  support 
extended  by  the  Air  Force  Office  of  Scientific  Research  under  Grant  AF-AFOSR 
82-0135A. 

The  principal  investigator  was  Professor  Sanjoy  Hitter.  The  contract 
monitors  were  Dr.  J.  Barns  and  Dr.  Hare  Jacobs  of  the  AFOSR  Directorate  of 
Hathematical  and  Information  Sciences. 

This  research  is  concerned  with  fundamental  aspects  of  filtering 
theory,  statistical  signal  processing  and  stochastic  variational  problems 
related  to  estimation  of  Harkov  Random  Fields.  We  take  a  viewpoint  which 
exploits  the  analogies  between  these  problems  and  problems  of  quantum  and 
statistical  physics.  The  proposed  research  is -of  great  potential  benefit  to 
the  U.S.  Air  Force  in  areas  such  as  guidance  and  control,  pattern 
recognition  and  image  processing  related  to  radar  signals  and  signal 


processing . 
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1.  Introduction 


J  This  report  is  concerned  with  the  following  fundamental  aspects  of 
Stochastic  Systems  Theory: 

) 

(i)  Filtering,  Statistical  Signal  Processing  and  related  Problems 

in  Scattering  and  Inverse  Scattering  Theory’ 

J 

Theory  of  Markov  Random  Fields  and  related  questions  in  Image 
Processing  and  Image  Understanding^ 

Stochastic  Variational  Calculus  and  Stochastic  Adaptive 
Control j  ’ 

(iv)  '  Parallel  and  Distributed  Algorithms  for  Statistical  Signal 
Processing  <-■ - - ' 


(ii) 


(iii) 


Problem  areas  (i)  and  (iii)  are  intimately  related  in  the  sense  that 
stochastic  control  in  the  presence  of  incomplete  observations  has  as  a  sub¬ 
problem  non-linear  filtering.  Moreover,  the  problem  of  parameter 
identification  could  be  considered  as  a  special  case  of  nonlinear  filtering 
and  the  problem  of  optimal  adaptive  control  (suitably  formulated)  can  be 
considered  as  a  special  case  of  stochastic  control  with  partiil  uncertain 
observat ions .  One  must  remark  that  this  necessitates  taking  a  Bayesian 
point  of  view. 

Stochastic  Calculus  of  Variations  could  be  considered  as  a  special  case 
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of  Stochastic  Control.  This  is  well  known  in  the  deterministic  situation 
and  indeed  the  two  fields  could  be  considered  equivalent,  namely,  by 

appropriate  transformation  one  can  pass  from  one  formulation  to  the  other. 

> 

This  is,  however,  not  so  in  a  stochastic  setting  where  careful  distinction 
needs  to  be  made  between  'open— loop  control*  (pre-programmed  control)  and 
'feedback  control'  (control  based  on  past  history  of  the  observations). 

This  report  describes  work  which  conforms  to  the  comprehensive  proposal 
submitted  to  the  Air  Force  Office  of  Scientific  Research  in  September  1981. 
One  of  the  new  directions  proposed  was  to  develop  a  Scattering  and  Inverse 
Scattering  Framework  for  estimation  problems  for  random  fields. 
Considerable  progress  has  been  made  in  the  direction  during  the  current 
period  of  the  grant.  In  addition,  during  the  current  period  of  this  grant 
we  have  initiated  new  research  on  Markov  Random  Field  models  for  images  for 
the  purpose  of  reconstruction  of  surfaces  from  noisy  data.  Using  a  Bayesian 
point  of  view,  these  estimation  problems  lead  to  stochastic  variational 
problems  which  can  be  solved  using  methods  of  simulated  annealing.  We  have 
undertaken  fundamental  theoretical  research  with  a  view  to  understanding  the 
method  of  simulated  annealing.  These  methods  are  also  particularly  suited 
for  parallel  and  distributed  computation. 

It  should  also  be  mentioned  that  the  work  proposed  here  is  potentially 
of  great  benefit  to  the  U.S.  Air  Force.  Increasingly,  it  is  being 
recognized  that  ad  hoc  techniques  using  linearization  and  perturbation 
methods  are  unsatisfactory  and  nonlinear  theory  is  ripe  for  applications. 
The  Kalman  filter  has  played  an  important  role  in  guidance  and  control  of 
aerospace  vehicles.  However,  the  extended  Kalman  filter  which  is  used  to 
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handle  nonlinear  situations  is  not  understood  from  a  scientific  point  of 
view  and  often  given  rise  to  incurable  convergence  difficulties.  We  have 
made  some  progress  towards  alleviating  this  situation.  In  particular,  we 
have  greatly  enhanced  understanding  about  the  derivation  and  functioning  of 
the  Extended  Kalman  Filter. 

The  control  of  future  aircrafts,  large  space  structures,  and  aerospace 
vehicles  is  a  problem  of  continuing  importance  from  the  point  of  view  of 
designing  adaptive  systems  that  operate  reliably  over  a  wide  operating 
envelope.  Similarly,  the  control  of  advanced  jet  engine,  whose  dynamic 
characteristics  change  rapidly  with  operating  conditions,  pose  difficult 
problems  if  one  wishes  to  design  a  control  system  which  accomplishes 
commanded  thrust  level  changes  rapidly,  while  maintaining  fan  and  compressor 
stability  margins.  It  would  appear  that  an  adequate  theory  of  stochastic 
adaptive  control  will  be  essential  to  solve  these  problems.  Due  to  the 
tremendous  increase  in  computing  power  available  and  decrease  in  costs  in 
memory  size  the  problem  of  dealing  with  non-linearities  is  no  longer  the 
insurmountable  obstacle  it  was. 

Signal  Processing  forms  an  important  aspect  for  various  systems  which 
is  vital  to  the  success  of  Air  Force  Missions.  Our  research  leads  to  new 
algorithms  and  new  computational  structures  for  the  solution  of  signal 
processing  problems.  Moreover  these  algorithms  and  structures  are 
particularly  suited  to  VLSI  implementations  and  indeed  often  suggest  novel 


VLSI  structures. 


• .  Fundamental  Viewpoint  of  oar  Research  and  Progress  to  Date 
2.1  Introdact ion 

Oar  research  is  concerned  with  the  fundamental  problem  of  statistical 

* 

signal  analysis,  namely,  the  optimal  probabilistic  reconstruction  of  a 
signal  from  noisy  observations  of  the  signal.  The  term  signal  analysis 
encompasses  snch  tasks  as  optimal  processing,  analyzing,  and  understanding 
of  signals. 

The  distinctive  features  of  this  research  are  (i)  the  modelling  and 
analysis  of  the  signals  in  a  form  which  has  striking  analogies  to  models  in 
statistical  and  quantum  physicsi  (ii)  the  optimal  incorporation  of  a  priori 
knowledge  about  the  signal  in  the  mathematical  modelt  (iii)  recognition  of 
the  role  of  symmetries  (in  particular,  the  relation  between  physical 
symmetries  and  the  action  of  a  symmetry  group  on  the  space  of  signals))  (iv) 
algorithms  for  probabilistic  reconstruction  and  analysis  which  permits 
implementation  in  parallel  and  distributed  architectures. 

The  identification  of  probabilistic  models  of  signals  as  models  arising 
in  physics  often  suggests  a  parallel  VLSI  (very  large  scale  semi-conductor 
integration)  implementation  of  the  algorithm  for  the  reconstruction  of  the 
signal.  Indeed,  it  is  believed  that  algorithms  that  are  best  for  VLSI 
implementation  must  mirror  the  physics  of  large-scale  semi-conductor 
integration.  An  example  of  this  process  might  be  the  modelling  of 
Stationary  Gaussian  Processes  as  generalized  transmission  lines. 


Our  recent  research  is  also  motivated  by  the  work  of  Marr  and  Poggio 
[1]  on  vision  and  has  as  its  aim  the  construction  of  a  mathematical  theory 
of  early  vision.  According  to  this  theory,  early  vision  is  primarily 


computational  and  is  concerned  with  such  functions  as  recognition  of 
symmetries,  surface  reconstruction,  edge-detection,  and  extraction  of  depth 
information.  The  attempt  at  a  construction  of  a  mathematical  theory  of 
early  vision  may  itself  shed  new  light  upon  theories  of  vision. 

2.2  The  Analogies  Between  Problems  in  Statistical  (and  Quantum)  Physics  and 

Optimal  Probabilistic  Reconstruction  of  Signals 

In  our  earlier  work  ([2],  [3],  [4])  we  have  shown  how  mathematical 
problems  of  linear  and  non-linear  filtering  (where  the  signals  are  modelled 
as  functions  of  Markov  diffusion  processes)  are  closely  related  to 
mathematical  problems  in  quantum  physics.  The  fundamental  reason  for  this 
is  that  the  representation  of  the  estimate  of  the  signal  (from  noisy 
observations)  as  a  conditional  expectation  can  be  achieved  using  a 
Stochastic  Feynman-Kac  formula.  The  recursive  estimation  problem  leads  to  a 
stochastic  partial  differential  equation  which  has  the  interpretation  as  a 
(euclidean)  field  and  in  this  viewpoint  the  Kalman  filter  plays  the  role  of 
the  free  euclidean  field  (Ornste  in-Ohlenbeck  operator)  of  quantum  physics. 
This  viewpoint  has  led  to  a  deepening  of  our  understanding  of  non-linear 
filtering  and  new  results. 

We  would  like  to  argue  that  this  viewpoint  of  analogies  with  physics  is 
deep  and  leads  to  new  insights  and  to  solutions  of  new  problems  in  the 
reconstruction  of  signals. 


2 . 3  Images  as  Markov  Random  Fields 

The  use  of  Markov  Random  fields  and  Bayesian  estimation  for  signal 


processing  tasks  sack  as  image  restoration  and  surface  reconstrnction  has 
recently  been  proposed  by  Grenander,  Geman  (cf.  for  example  [5])  and  our  own 
unpublished  work.  The  simplest  such  model  of  the  signal  (image  to  be 
recovered)  corresponds  to  a  one-dimensional  Ising  ferromagnet  on  a  finite 
lattice  with  free  boundary  conditions.  If  such  a  signal  is  observed  in  the 
presence  of  noise,  then  the  corresponding  mathematical  model  corresponds  to 
an  Ising  ferromagnet  with  an  external  random  magnetic  field.  The  signal-to- 
noise  ratio  has  the  interpretation  as  the  temperature  of  the  system.  Given 
the  prior  distribution  of  the  signal  and  the  noise,  the  posterior 
distribution  can  be  computed  using  Bayes  formula  and  is  in  the  form  of  a 
Gibbs  distribution.  The  maximum  a  posteriori  probability  estimate  can  be 
reduced  to  the  minimization  of  an  energy  function  and  corresponds  to  finding 
the  ground  state  of  the  Ising  ferromagnet  with  external  field.  This  problem 
is  effectively  a  large  integer-quadratic  programming  problem  where  the 
matrix  has  a  band  structure. 

In  recent  work  we  have  been  able  to  decompose  this  problem  into  a 
sequence  of  one-dimensional  minimization  problems  by  using  a  dynamic 
programming  recursion  on  the  boundaries  (odd  bonds  between  neighboring 
cells).  This  decomposition  corresponds  to  solving  a  sequence  of  estimation 
problems  under  different  "scales*  and  is  strikingly  resemblant  of  the 
renormalization  group  approach  to  statistical  mechanics  developed  by  K. 
Wilson.  In  contrast  to  physics,  however,  here  we  are  interested  in 
obtaining  the  detailed  structure  of  the  random  fields. 

We  have  also  used  Markov  random  field  models  in  conjunction  with 
stochastic  approximation  (simulated  annealing  in  the  terminology  of 


Kirkpatrick,  et  al.  [6])  for  surface  reconstruction  that  preserves 
discontinuities  in  images. 

2.4  The  S-Matrix  and  Estimation  for  Stationary  Gaussian  Processes 

To  illustrate  the  power  of  this  viewpoint,  we  discuss  some  of  our 
recent  work  on  modelling  and  estimation  of  stationary  Gaussian  processes 
indexed  by  the  integers.  The  basic  idea  is  that  the  analog  of  the  S-matrix 
(scattering  function)  representing  the  interaction  between  the  past  and 
future  of  a  regular  stationary  Gaussian  process  is  the  fundamental  object 
for  performing  estimation,  prediction,  and  interpolation  for  Gaussian 
signals  in  additive  white  noise.  Indeed  the  S— matrix  can  be  explicitly 
computed  and  is  an  L°“-function,  unitary  on  the  boundary  of  the  unit-disc. 
If  the  process  is  strongly  mixing,  then  this  function  is  in  the  class  H®  + 
C.  Under  some  further  mild  assumptions,  it  can  be  shown  that  the  S-matrix 
uniquely  characterizes  the  spectral  density  of  the  process  (up  to  a 
multiplicative  constant).  The  relationship  of  this  S-matrix  to  the 
scattering  function  of  Lax  and  Phillips  can  be  explicitly  characterized. 

The  Hankel  and  Toeplitz  operators  induced  by  the  S-matrix  have  a 
special  role  to  play  in  this  theory,  and  using  these  operators,  one  can 
solve  the  filtering  and  prediction  problem  and  leads  to  a  new  formulation  of 
the  approximation  of  Gaussian  processes  [7],  This  work  uses  the  deep  theory 
of  approximation  of  Hankel  operators  as  developed  by  Adamjan,  Arov  and 
Krein  [8] . 


2.5  Estimation  Theory,  Statistical  Signal  Processing  and  Inverse  Problems 


One  of  the  objectives  of  our  research  has  been  to  study  the  links 
existing  between  linear  estimation  theory  and  inverse  scattering  theory,  and 
to  use  this  relationship  to  obtain  efficient  algorithms  for  solving  inverse 
scattering  problems.  The  algorithms  that  we  have  obtained  are  efficient, 
recursive,  and  operate  on  a  layer  stripping  principle,  whereby  an  unknown 
scattering  medium  is  reconstructed  layer  by  layer,  in  a  sequential  fashion. 
These  algorithms  can  be  viewed  as  the  counterpart  of  the  celebrated  fast 
algorithms  of  linear  estimation  theory  due  to  Levinson,  Krein,  Szego,  and 
Schur,  which  have  been  used  extensively  by  Kailath  and  his  colleagues, 
Dewilde,  Dym,  and  many  others.  More  recently,  we  have  also  been  able  to 
show  that  Kalman  filtering  techniques  can  be  applied  to  inverse  scattering 
problems,  when  reflection  coefficients  have  a  rational  structure.  The 
inverse  scattering  methods  that  were  obtained  by  this  analogy  with  linear 
estimation  theory  were  then  applied  to  several  inverse  problems  such  as  the 
inverse  seismic  or  inverse  resistivity  problems  of  geophysics.  The  study  of 
the  relations  existing  between  linear  estimation  and  inverse  scattering  has 
been  beneficial  not  only  to  inverse  scattering,  but  also  to  linear 
estimation,  where  we  have  been  able  to  solve  several  previously  unsolved 
problems.  For  example,  in  [6]  by  using  the  analogy  existing  between  the 
linear  estimation  problem  for  isotropic  random  fields,  and  the  two- 
dimensional  ( 2— D )  inverse  scattering  problem  for  a  potential  with  radial 
symmetry,  we  have  obtained  an  efficient  estimation  technique  for  isotropic 
fields.  This  technique  can  be  viewed  as  a  generalization  of  the  1-D 
Levinson  equations  of  linear  prediction,  but  previous  attempts  to  extend  the 
Levinson  recursions  to  two-dimensions  had  failed  because  they  were  forcing  a 
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quarte r— p lane  or  half-plane  causality  structure  which  does  not  exist  for 


random  fields. 

Since  our  results  are  described  in  detail  in  [l]-[9],  we  will  only 
outline  here  the  main  aspects  of  our  work. 

(1)  In  the  area  of  inverse  scattering,  we  have  obtained  in  [4],  [5]  a 
new  class  of  differential  inverse  scattering  methods  which  operate 
on  a  layer  stripping  principle  and  reconstruct  a  scattering  medium 
recursively,  layer  by  layer.  These  inverse  scattering  methods 
generalize  an  algorithm  introduced  by  Schur  in  1917  for  testing 
the  boundedness  of  a  function  which  is  analytic  inside  the  unit 
circle.  The  recursions  appearing  in  this  algorithm  are  also 
identical  to  the  so-called  fast  Cholesky  equations  for  factoring  a 
Toeplitz  operator  in  causal  times  anticausal  (or  lower  times  upper 
triangular)  form.  From  a  more  general  point  of  view,  differential 
inverse  scattering  algorithms  rely  on  the  method  of  propagation  of 
s ingularities ,  and  depending  on  whether  this  method  is  applied  to 
two-component  wave  equations,  to  the  telegrapher's  equations,  or 
to  Schrodinger  equations,  layer  stripping  algorithms  can  be 
expressed  in  several  forms.  Thus  in  addition  to  the  Schur 
algorithm  that  we  have  developed,  several  variations  of  the  same 
technique  have  been  proposed  by  Symes,  Santosa  and  Schwetlick, 
Bube  and  Burridge,  and  Corones  and  his  group,  among  others. 
Because  of  their  recursive  structure,  and  because  the  quantities 
that  they  propagate  can  be  interpreted  physically  as  being  the 
waves  inside  the  medium,  layer  stripping  algorithms  are  more 


convenient  than  traditional  inverse  scattering  methods  (introduced 
by  Gelfand  and  Levitan,  Marchenko,  Krein,  Kay  and  Moses, 
Faddeev...)  which  rely  on  integral  equations.  However,  it  was 
shown  in  [4]  that  differential  and  integral  equations  methods  can 
be  related  from  a  system-theoretic  point  of  view,  by  using 
causality.  In  [5]  various  applications  of  layer  stripping 
algorithms  to  the  reconstruction  of  transmission  lines,  inverse 
seismic  problems,  and  linear  estimation  of  stationary  processes 
are  discussed.  In  addition,  for  the  case  when  the  scattering 
medium  that  we  want  to  reconstruct  is  lossy,  and  when  we  have 
access  to  transmission  data,  as  well  as  scattering  data,  a 
generalization  of  the  Schur  algorithm  based  on  two  sets  of  coupled 
equations  is  derived  and  is  used  to  reconstruct  a  lossy 
transmission  line. 

(2)  In  [7]  the  inverse  scattering  problem  for  the  case  when  the 
reflection  coefficient  of  the  scattering  medium  is  rational  was 
considered.  Several  solutions  of  this  problem  have  been  proposed 
in  the  past,  but  all  these  solutions  were  quite  inefficient  since 
they  relied  on  exploiting  the  rational  structure  of  the  reflection 
coefficient  inside  a  Viener  Hopf  equation  defined  for  every  value 
of  the  depth  x  at  which  the  reflectivity  function  r(x)  of  the 
scattering  medium  needs  to  be  reconstructed.  The  missing  concept 
was  clearly  that  of  a  state-space  model.  In  [7]  by  using  a  simple 
state-space  representation  for  the  left  and  right  going  waves 


propagating  inside  the  medium  when  the  medium  is  probed  from  the 
left  by  an  impulsive  wave,  we  were  able  to  obtain  a  Kalman  filter¬ 
like  solution  for  the  inverse  scattering  problem.  This  solution 
is  expressed  in  terms  of  the  so-called  Chandrasekhar  equations  of 
linear  filtering  theory. 

(3)  Since  the  fast  inverse  scattering  algorithms  for  two-component 
wave  systems  described  in  [4],  £5}  are  expressed  in  terms  of  the 
Levinson  and  fast  Cholesky  recursions,  and  since  these  recursions 
are  usually  associated  with  a  Gram-Schmidt  orthonormalization 
process,  and  perform  a  factorization  of  a  Toeplitz  (or  Hankel) 
operator  or  of  its  resolvent  in  terms  of  triangular  operators,  it 
is  natural  to  ask  whether  a  similar  orthonormalization  point  of 
view  can  be  used  to  interpret  the  results  of  [4],  [5],  In  [91  it 
is  shown  that  such  an  interpretation  exists,  but  that  unlike  in 
the  derivation  of  the  Szego  orthogonal  polynomials,  or  of  the 
continuous  Krein  polynomials,  where  a  scalar  spectral  function  was 
used,  we  need  to  associate  a  2x2  matrix  spectral  function  to  a 
two-component  wave  system.  Then  by  orthonormalizing  the  free 
solut ions  of  the  system  with  respect  to  this  matrix,  we  obtain  the 
Marchenko  equations  for  solving  the  inverse  scattering  problem. 
The  functions  obtained  by  this  Gram-Schmidt  orthonormalization 
process  are  now  2x2  matrix  orthonormal  functions,  and  all  sorts  of 
identities,  such  as  the  Chr istof fe 1-Darboux  formulas,  can  be 
obtained  for  these  functions.  The  matrix  operators  factored  by 


the  Levinson  and  fast  Cholesky  recnrsions  are  also  pat  in 
evidence.  Ve  hope  that  these  results  will  be  useful  in  the 


context  of  linear  estimation  to  obtain  some  further  insight  on  the 
properties  of  lattice  realizations  of  scalar  stationary  stochastic 
processes . 

(4)  The  layer  stripping  inverse  scattering  methods  developed  in  [4], 
[5]  were  used  in  [l]-[3]  to  solve  the  inverse  seismic  problem  for 
one-dimensional  layered  media.  In  [1],  the  case  of  an  acoustic 
medium  was  considered.  In  this  problem,  the  objective  is  to 
reconstruct  both  the  density  p{x)  and  the  wave  speed  c(x)  of  the 
medium  as  functions  of  the  depth  x.  The  Schur  algorithm  was  used 
to  solve  this  problem.  By  probing  the  medium  obliquely  at  two 
different  angles  with  plane  waves,  and  by  running  two  sets  of 

recursions  corresponding  to  these  two  experiments,  it  was  shown 
that  the  density  and  wave  speed  could  be  reconstructed 
simultaneously,  in  a  recursive  way,  by  increasing  depths.  The 

point  source  case  (i.e.  the  case  when  the  probing  wave  is  an 
impulsive  spherical  wave)  was  also  considered.  In  this  case,  it 
was  shown  that  by  slant-stacking,  i.e.  by  performing  a  Radon 

transform  on  the  data  collected  by  the  receivers  on  the  surface, 
the  plane-wave  data  could  be  synthesized,  and  the  algorithm 

mentioned  above  for  two  plane  waves  could  be  used.  In  [2],  the 
more  difficult  problem  when  the  medium  is  described  by  the 
equations  of  elasticity  was  considered.  In  this  case  two  types  of 
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waves,  the  pressure  (P)  and  stress  (S)  waves,  can  propagate  inside 
the  medium  and  can  be  converted  into  each  other  at  interfaces. 

The  objective  is  to  reconstruct  the  two  Lame  parameters  X(x)  and 

* 

H ( x )  of  the  medium  and  the  density  p(x)  as  functions  of  depth.  By 
performing  two  experiments  with  impulsive  plane  P  and  S  waves 
which  are  obliquely  incident  upon  the  medium,  in  such  a  way  that 
the  lateral  wave  number  is  the  same  in  both  experiments,  and  by 

propagating  recursively  the  two  sets  of  measured  waves  for  these 

two  experiments  for  increasing  depth,  X(x),  p(x)  and  p(x)  can  be 
reconstructed  recursively.  The  only  difference  with  [1]  is  that 
the  system  used  to  propagate  the  downgoing  and  upgoing  P  and  S 
waves  is  a  differential  system  of  order  4  (instead  of  order  2  for 
the  acoustic  problem).  In  [3]  the  inverse  problem  for  an  acoustic 
medium  probed  by  spherical  harmonic  waves,  and  when  the  reflected 
data  is  measured  for  all  lateral  wave  numbers,  was  considered.  In 

this  case,  given  the  data  for  two  frequencies  (instead  of  two 

angles  in  [1])  and  by  using  a  transformation  procedure  to  solve 
the  inverse  resistivity  problem  (see  (5)  below),  it  was  shown  that 
a  layer  stripping  technique  could  be  used  to  reconstruct  the 
density  p(x)  and  wave  speed  c(x)  separately. 

(5)  In  [8]  the  inverse  resistivity  problem  for  a  layered  earth  was 
considered.  In  this  problem,  the  earth  is  probed  by  injecting 
some  current  inside  the  earth,  and  lateral  potential  gradients  on 
the  surface  ae  measured.  This  problem  was  suggested  to  us  by  Dr. 


Steve  Lang  from  Shlumberger .  Since  the  potential  is  described  by 
an  elliptic  equation,  and  not  by  the  wave  equation,  this  problem 
is  not  an  inverse  scattering  problem.  Nevertheless ,  there  exists 
a  simple  mapping  technique  which  can  be  used  to  convert  the 
inverse  resistivity  problem  into  an  equivalent  inverse  scattering 
problem,  to  which  existing  1-D  inverse  scattering  techniques  can 
then  be  applied.  In  this  context,  it  turns  out  that  a  number  of 
previously  introduced  inverse  resistivity  techniques  can  be 

identified  with  classical  signal  processing  algorithms.  The 
mapping  used  to  transform  the  inverse  resistivity  problem  into  an 
equivalent  inverse  scattering  problem  has  also  an  elegant 

interpretation  in  terms  of  Maxwell's  method  of  images. 

(6)  In  [6]  the  problem  of  estimating  a  2-D  isotropic  random  field  in 

noise  given  some  observations  over  a  disk  of  radius  R  was 

examined.  The  isotropy  of  the  field  and  the  circular  symmetry  of 
the  observation  geometry  were  exploited  by  expanding  the  field  in 
Fourier  series,  and  noting  that  the  Fourier  coefficient  processes 
were  uncorrelated.  This  implies  that  the  original  2-D  estimation 
estimation  problem  can  be  decomposed  into  a  countable  set  of  1-D 
estimation  problems  for  the  Fourier  coefficient  processes.  The 
filtering  equation  for  each  Fourier  coefficient  process  turns  out 
to  be  identical  to  the  Ge If and-Levitan  equation  for  reconstructing 
a  circularly  symmetric  potential  from  its  spectral  density  r(X), 
where  r(X)  is  here  the  spectral  density  of  the  random  field.  The 


17 


structure  of  the  Ge If and-Levitan  equation  can  then  be  exploited  to 
obtain  some  Levinson-like  recursions  for  the  optimum  filter,  which 
are  very  efficient,  and  provide  therefore  a  simple  solution  of  the 
random  field  estimation  problem.  This  algorithm  is  the  first  fast 
algorithm  ever  derived  in  two  dimensions,  and  a  preliminary  study 
indicates  that  it  can  also  be  used  to  solve  the  2-D  maximum 
entropy  spectral  estimation  problem  for  isotropic  fields. 

2.6  Nonlinear  Filtering  and  Stochastic  Variational  Problems 

A  long-standing  open  problem  in  filtering  theory  has  been  to  obtain  a 
derivation  of  the  Extended  Kalman  Filter  and  explain  its  qualitative 
behavior.  The  Extended  Kalman  Filter  is  widely  used  in  aerospace  systems 
and  is  known  to  function  very  well  in  many  situations,  but  is  also  known  to 
exhibit  divergence  phenomenon  in  the  presence  of  modelling  errors.  In  the 
paper  [1],  first  steps  towards  a  rigorous  derivation  of  Extended  Kalman 
Filter  as  well  as  explaining  its  qualitative  properties  were  taken.  To 
obtain  this  result,  we  use  the  stochastic  control  interpretation  of 
nonlinear  filtering  described  in  the  joint  paper  with  Wendell  Fleming  [2], 
the  Morse  Lemma  with  parameters  [3],  and  the  work  of  Malliavin  on  Stochastic 
Jacobi  fields  [4] . 

It  was  suggested  that  the  Stochastic  Control  Interpretation  of 
Nonlinear  Filtering  would  provide  the  means  for  obtaining  bounds  for 
nonlinear  filtering.  A  first  step  in  this  direction  has  been  taken  in  the 
paper  mentioned  above.  Specifically,  the  analog  of  the  Fisher  Information 
Matrix  has  been  defined.  This  paper  also  shows  how  the  nonlinear  filtering 
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problem  is  related  to  the  identification  problem.  Ve  have  done  fnrther  work, 
on  these  issues  and  we  are  considering  examples  related  to  the  phase-lock 
loop  to  understand  better  various  lower  bounds  on  performance  that  can  be 
obtained.  A  joint  paper  with  A.  Moro  and  J.  Moura  is  now  in  preparation. 
This  work  also  has  connections  with  recent  work  of  Bobrovsky  et  al  [5]  and 
Adaptive  Filtering. 

An  important  open  problem  for  the  last  few  years  has  been  the 
construction  of  robust  filters  using  pathwise  nonlinear  filtering  for 
observations  which  are  unbounded.  This  problem  has  been  settled  in  the 
negative  by  Sussmann  based  on  earlier  work  by  Mitter.  More  specifically,  we 
have  shown  by  example  that  the  normalization  constant  becomes  infinity  for 
certain  "physical"  observation  paths.  This  suggests  that  the  normalization 
is  important  and  it  is  necessary  to  construct  the  normalized  conditioned 
density  by  examining  the  robust  versions  of  the  numerator  and  denominator  of 
the  Kallianpur-Striebel  Formula  after  "cut-offs'  have  been  introduced.  One 
then  has  to  remove  the  cut-off  in  an  appropriate  way.  This  is  a  familiar 
procedure  in  Statistical  Mechanics  but  some  key  monotonicity  properties  seem 
to  be  absent  in  the  problem  under  investigation. 

For  further  progress  on  the  qualitative  behavior  on  nonlinear  filters, 

it  is  necessary  to  understand  the  small-time  and  large-time  behaviors  of 

these  filters.  Mathematically,  this  problem  is  related  to  the  small-time 

behavior  of  conditional  diffusions  and  to  large  deviation  theory.  For 

diffusion  processes  there  now  exists  a  large  body  of  theory  developed  by 

Donsker-Varadhan  and  Ventce 1-Fre id  1  in .  What  is  needed  is  a  generalization 

of  these  ideas  to  conditional'  diffusion  processes,  and  we  have  begun 
work  on  that  generalization. 
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